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Abstract Body 



Background / Context: 

In canonical applications of the instrumental variable method, exogenously determined 
exposure to an instrument (sueh as random assignment to a treatment condition) induces 
exposure to a mediating proeess that in turn eauses a change in a later outcome. A crucial 
assumption known as the exelusion restriction is that the hypothesized instrument ean influenee 
the outcome only through its influenee on exposure to the mediator of interest (Heckman & 
Robb, 1985; Imbens & Angrist, 1994). It may be the case, however, that multiple mediators 
operate jointly to influence the outcome, in whieh case a single instrument will not suffiee to 
identify the eausal effeets of interest. 

To cope with this problem, analysts have recently exploited the fact that a causal process 
is often replicated aeross multiple sites, generating the possibility of multiple instruments in the 
form of site-by-instrument interactions. These multiple instruments ean, in principle, enable the 
investigator to identify the impaet of multiple programs or treatments regarded as the mediators 
of the effect of an instrument. Kling, Liebman, and Katz (2007), for example, used random 
assignment in the Moving to Opportunity (^TO”) study as an instrument to estimate the impact 
of neighborhood poverty on health, soeial behavior, education, and economic self-sufficiency of 
adolescents and adults. Reasoning that the instrument might affect outcomes through 
meehanisms other than neighborhood poverty, they eontrol for a second mediator, use of the 
randomized treatment voucher. To do so, they capitalize on the replication of the MTO 
experiment in five cities, generating ten instruments (the five sites generate ten site -by-treatment 
interaetions as instruments beeause there were three randomly assigned treatment conditions per 
site) to identify the impaet of the two mediators of interest, neighborhood poverty and 
experimental compliance. Using a similar strategy, Morris, Dunean, and Rodriguez (2010) used 
data from 16 implementations of welfare-to-work experiments to identify the impact of family 
income, average hours worked, and reeeipt of welfare as mediators. 

Clearly, this strategy for generating multiple instruments has potentially great appeal in 
research on causal effects in social science. For example, Spybrook (2009) found that, among 75 
large-scale experiments funded by the US Institute of Education Sciences over the past deeade, 
the majority were multi-site studies in whieh randomization oceurred within sites. In prineiple, 
these data could yield a wealth of new knowledge about causal effects in education policy. It is 
essential, however, that researchers understand the assumptions required to pursue this strategy 
sueeessfully. To date, we know of no eomplete account of these assumptions. 



Purpose / Objective / Research Question / Focus of Study: 

Our purpose is to clarify the assumptions that must be met if this — mulple site, multiple 
mediator” strategy, hereafter referred to as ^SMM,” is to identify the average causal effects 
(ATE) in the populations of interest. 
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Setting: 

N/A 

Population / Participants / Subjects: 

N/A 

Intervention / Program / Practice: 

N/A 



Significance / Novelty of study: 

We are aware of only two studies that rely on site -by-treatment interactions to generate 
multiple instruments to estimate the impacts of multiple potential mediators (Kling et ah, 2007; 
Morris et al, 2010). Neither of these studies discusses the assumptions underlying the models 
they fit. Given the large number of multi-site randomized trials that have been conducted in 
education and in other fields (Spybrook, 2009), such models are likely to become increasingly 
appealing as a means to estimating the effects of multiple hard-to-randomize potential 
mechanisms. Ours is the first paper that systematically describes the identifying assumptions of 
such models. 

Statistical, Measurement, or Econometric Model: 

We begin by delineating the assumptions required for identification in the case of a single 
instrument and a single mediator within a single-site study. Unlike Angrist, Imbens, and Rubin 
(1996) (hereafter AIR), we consider the general case where the mediator may be continuous or 
multi-valued. In this general case, the assumptions required for identification of the average 
treatment effect differ somewhat from those AIR (1996) describe for the binary mediator case. 
We link our discussion to recent papers describing the correlated random coefficient model, and 
show that the CRC model is identified by instrumental variables using a weaker assumption than 
that described by Heckman and Vytlacil (1998) and Wooldridge (2003). 

Following a discussion of the single, site, single mediator case, we then consider the case 
of multiple sites with a single mediator before turning to the case of primary interest: MSMM. 
Finally, we generalize these results to any setting in which multiple instruments identify the 
impact of multiple mediators. 



Usefulness / Applicability of Method: 

Researchers often want to understand the mechanisms through which a specific 
treatment, program, or policy operates. Although it may be feasible to randomize individuals to 
specific treatments or programs, it is often not feasible to assign individuals to processes that are 
hypothesized to be mediators. For example, in the Reading First Impact Study (Gamse et ah. 
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2008), a regression diseontinuity design provided exogenous variation in Reading First study, 
enabling the researchers to estimate the effects of the Reading First program as a whole. 
However, the program was hypothesized to work through its impacts on five distinct dimensions 
of teachers’ reading instruction practices. These practices could not be randomly assigned, 
however (because the researchers cannot control what teachers do in the classroom). One could 
imagine using the site -by-instrument interactions in an IV model to estimate the effects of 
specific instructional practices. There are many such cases in educational and social science 
research. Thus, a clearer understanding of the necessary assumptions (and the consequences of 
their failure) may improve the quality of research on hard-to-randomize educational processes 
and mechanisms. 



Research Design: 

We consider the case of a study design in which there are multiple sites, indexed by s G 
(1,2, ... ,/}. In each site s, individuals are assigned to one of multiple possible values of a 
treatment, T, where T may be binary or continuous. The treatment T may affect a vector of P 
mediators, M = [M^, M 2 , ... , Mp}] each of the mediators may also be binary or continuous. We 
describe the person-specific effect of T on a mediator Mp as the person-specific -eompliance” 
with respect to mediator Mp. Each of the mediators may each affect an outcome, Y. We are 
interested in estimating the average effect of each of the mediators on the outcome Y. 



Data Collection and Analysis: 

N/A 

Findings / Results: 

We identify 9 assumptions that must be met in order that the MSMM IV model identifies the 
average effects of each mediator in the population of interest. These are: 

1 . Stable unit treatment value assumption (SUTVA): each unit has one and only one 
potential outcome under each treatment condition: that is, for a population of size n, 
Tj(ti, t 2 , ... , tji) = Tj(tj) for all i G (1,2, ... , n). In the IV model, this standard SUTVA 
assumption is actually composed of two distinct SUTVA assumptions: 

a. For each mediator p, each unit i has one and only one potential value of Mpi for 
each treatment condition t: that is, for a population of size n, Mpi{t]^, t 2 , ... , t^j) = 
Mpi(ti) for all i G (1,2, ...,n). 

b. Each unit i has one and only one potential outcome value of Tj for each vector of 
mediators Mj = M 2 i , ... , Mpi}: that is, for a population of size n, 

Fi(Mi, M 2 M^) = Yi(Mi) for all i G {1,2 n). 
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2. Exclusion restriction : the treatment T affeets Y only through its impaet on the set of P 
mediators, M = {Mi, M 2 , ...,Mp). That is, T(t) = T(t,M(t)) = T(M(t)). 

3. Person-speeifie linearity of eaeh mediator in T : the person-speeifie effeet of T oneaeh 
mediator Mp is linear. That is, Mp(t) = Mp(0) + t p for eaeh p. 

4. Person-speeifie linearity in M : the person-speeifie effeet of eaeh mediator Mp is linear. 
That is, T(M(t)) = T(M(0)) + I^^i p(t p). 

5. Ignorable within-site treatment assignment : the assignment of the instrument (the 
treatment, in our notation) must be independent of the potential outeomes within eaeh 
site: T 1 T(t)|s, T 1 M(t)|s,V t,s. 

6. No average within-site eomplianee-effeet eovarianee : E ^CoVcf p, p)j = O.Vp G 

(1,2, ... , P}. A simpler, but stronger, assumption is that there is no within-site 
eomplianee-effeet eovarianee in any site: CoVs( p, p) — 0,Vp E (1,2, ...,P], Vs G 
(1,2 5). 

7. Site -by-mediator eomplianee matrix has suffieient rank . In partieular, if G is the S xP 
matrix of the Ysp then rank^G) = P. This implies three speeifie eonditions: 

a. The eomplianee of at least P — 1 of the mediators varies aeross sites. That is, 
Var(Yps^ — 0, for at most one p G (1,2, ... , P). 

b. There are at least as many sites as mediators: P < S. 

e. There is some subset of Q site-speeifie average eomplianee veetors, = 
iVis’ Yis’ — • Yps\^ where S > Q > P, that are linearly independent. 

8. Parallel mediators : assignment to T does not influenee a given mediator Mp through any 

other mediator M^, q G (1,2, ... , P). That is, the mediators do not influenee one another. 
That is, Mp(t,M^, ...,Mp_i,Mp+i, = Mp(t),Vp G (1,2, , ... , P). 

9. Mean independenee of the site-average eomplianees and effeets . Within eaeh site s, let 

Yps = E[ pl'^] and Spg = E[ p|s] be the site-average eomplianee and effeet of mediator 
p, respeetively. Likewise, let Yp — E[ p] and Sp — E[ p] be the average eomplianee 
and average effeet of mediator p in the population. Then we assume that the site-average 
effeets are independent of the site average eomplianees. That is, p[5ps|Tis, Y 2 s> — > Tps] — 
E[Sp,]^SpypE{i p). 

Some of these are familiar assumptions from the standard IV model in the ease where 
there is a single site (J = 1) and a single mediator (P = 1). In this ease, assumptions 1-7 are 
suffieient for the IV model to identify the average effeet of the mediator on Y in the population. 
Assumptions 8 and 9 pertain only when there are multiple mediators (assumption 8) or multiple 
sites (assumption 9). Moreover, in this ease, if both T and M are binary, then assumptions 3 and 
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4 are unnecessary; assumptions 1, 2, 5, 6 and 7 are sufficient to identify the average effect of M 
on Y in the population. 

We note that assumptions 1, 2, 5, and 7 are generalizations of four of the five IV 
assumptions identified by Angrist, Imbens, and Rubin (1996) [note that in the case when 

5 — P — 1, the sufficient rank assumption is equivalent to the ^lonzero average causal effect of 
T on M” assumption made by AIR], Their fifth assumption, montonicity, is notably absent from 
our set of assumptions, however. We show that the monotonicity assumption is relevant only in 
the case where both T and M are binary and where we wish to identify the average effect of M 
on Y among the population of -eompliers”, in Angrist, Imbens, and Rubins’ (1996) terminology. 
If we wish to identify the average effect of M on T in the population (arguably an estimand of 
more general interest), then no montonicity assumption is needed. Instead, we require 
assumption 6, the no compliance-effect covariance assumption. Moreover, this assumption is 
relevant even when T and M are not binary. Thus, our set of assumptions is more generally 
applicable than the AIR assumptions, and identifies an estimand that is of more general interest. 

We note also that assumption 6 (no within-site compliance-effect covariance) is weaker 
than the assumption that Heckman and Vytlacil (1998) and Wooldridge (2003) make in order to 
identify average effects in the correlated random coefficients model. We show that the 
assumption of no compliance effect covariance (along with the other standard IV assumptions) is 
sufficient to identify the average effects using an IV model when the data are generated from a 
correlated random effects process. 



Conclusions: 

Our investigation of the assumptions of the multiple -mediator, multiple-site IV model 
demonstrates that such models rely on a large number of non-trivial assumptions. Of most 
importance are the assumptions regarding the relationship between compliance and effect. Any 
correlation — ^whether within or between sites — between compliance (with respect to any 
meditator) and effect (again, of any mediator) will potentially bias the estimates of the effect of 
any mediator. Because individuals or groups (sites) with the most to gain from a specific 
practice (mediator) may be more likely to comply with treatment assignment, there is good 
reason to worry that compliance-effect covariances are non-zero in many cases. 
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